Papers with subword language models
Adversarial Subword Regularization for Robust Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for segmenting words into subword units are not robust enough to handle multiple subword candidates. |
| Approach: | They propose to regularize subword segmentations that maximize the translation loss by using gradient signals during training to prevent erroneous segmentations of unseen words. |
| Outcome: | The proposed method improves the performance of NMT models on low-resource and out-domain datasets. |
Learning Mutually Informed Representations for Characters and Subwords (2024.findings-naacl)
Copied to clipboard
| Challenge: | Pretrained language models rely on subword tokenization to process text as a sequence of subwords. |
| Approach: | They propose a character-subword language model that integrates character and subword modalities into one model. |
| Outcome: | The proposed model outperforms its backbone language models on English sequence labeling and classification tasks. |
Subword Language Model for Query Auto-Completion (D19-1)
Copied to clipboard
| Challenge: | Current neural query auto-completion systems rely on character-level language models but they slow down when queries are long. |
| Approach: | They propose to use subword language models for query auto-completion . they propose to implement a retrace algorithm and a method for approximate marginalization . |
| Outcome: | The proposed model achieves 2.5 times faster while maintaining similar quality to character-level baseline. |